Many industrial robots are operated based on a scheme called teaching playback. That is, the movement and operations the user wants the robot to perform have been taught in advance as a program so as to make the robot perform desired operations by playing back the program. The robot is able to repeat programmed movement and operations exactly. However, once the object is placed at a position different from that taught by the program, the robot cannot perform correct operations on the object.
To deal with this, there is a known technology in which the position of the object is recognized by use of a vision sensor such as a camera, etc., so that the operation of the robot is corrected based on the positional information on the object recognized by the vision sensor. This method makes it possible to perform exact operations on the object even if the object is placed at a position different from that taught by the program.
The position of the object identified by the vision sensor is represented in a coordinate system. (which will be referred to hereinbelow as “the vision coordinate system”) based on which measurement of the vision sensor is performed. In order to correct the operation of the robot based on the positional information identified by the vision sensor, the positional information represented in the vision coordinate system needs to be transformed into the positional information in the coordinate system (which will be referred to hereinbelow as “the robot coordinate system”) based on which the operation of the robot is controlled. The data for converting the positional information represented in the vision coordinate system into the positional information represented in the robot coordinate system is referred to as “camera calibration data”, and the job for acquiring the camera calibration data is referred to as “camera calibration” (e.g., Japanese Unexamined Patent Publication No. JP-A-10-49218).
On the other hand, there is a known technique called visual feedback as a method that enables correction of the position of the robot using a vision sensor without performing camera calibration (e.g., Japanese Unexamined Patent Publication No. JP-A-2003-211381, which will be referred to hereinbelow as “Patent Document 2”). For example, the robot is moved to a position for gripping the object and is made to take an image of the object by the camera attached to the robot and store the appearance of the object as target data. In the actual operation, an image of the object is taken by the camera every time the robot is moved, then the position and attitude of the robot is controlled so that the appearance of the object coincides with the target data. This process is iterated. When the error has converged to zero, gripping with the hand is implemented.
In the method described in Patent Document 2, the robot grips the object by a hand at a position where the error has become zero. This means that the appearance of the object on the camera when the robot is positioned at a place where the object is gripped, needs to be stored as the target data. This requirement limits the positional relationship between the camera and the hand attached at the arm end of the robot, hence restricts free design.
Since this method enables the robot to perform operations on the object only at the position Where the error is zero, the operation is free from problems when the robot is made to perform a simple task such as gripping the object. However, this method cannot be used when it is necessary to make the robot perform a job that needs complicated motion paths such as to apply an adhesive to the object through a predetermined path, for instance.
There is another proposed method in which the difference between the position of the robot when target data was configured relative to a reference object and the position of the robot when the robot performs a gripping operation of the reference object, has been stored as a position transformation matrix, and the robot is operated to converge the error to zero, then is moved following the stored position transformation matrix to perform a gripping operation (e.g., Japanese Patent Publication No. JP-B-4265088). In this invention, in comparison with the method disclosed in Patent Document 2, restriction on the positional relationship between the camera and the hand attached to the arm end of the robot is eliminated. However, this method offers no solution to the problem that the method cannot be used when the robot is required to perform a job that needs complicated motion paths.
Incidentally, when the object is placed on a flat pedestal such as a table or the like, in order to control the position and attitude of the robot so that the appearance of the object coincides with the target data, it is preferable that the robot is controlled to keep the distance from the camera attached to the arm end of the robot to the table on which the object is placed, unvaried. Further, when the orientation of the object changes, it is necessary to control the robot so that the relative direction of the object to the camera is unchanged. For this purpose, it is preferable that the camera is controlled so as to rotate on an axis that is perpendicular to the table surface so that the inclination of the camera relative to the table will not change.
However, in the general visual feedback, it is neither guaranteed that the distance between the camera and the table is unchanged nor that the relative inclination between the camera and the table is unchanged. As a result, there are cases in which controlling the robot so that the appearance of the object coincides with the target data takes a longer time than needed.
In order to avoid this problem, a coordinate system that is parallel to the table surface may and should be configured in the robot so that the camera can be moved based on this coordinate system. For example, it is possible to configure a robot coordinate system on the table surface by defining an arbitrary point on the hand of the robot as the TCP (Tool Center Point), making the TCP of the robot touch three points on the table surface but not in a line to read the coordinate values of the TCP at each point, viewed from the robot coordinate system (e.g., Japanese Unexamined Patent Publication No. JP-A-2010-076054).
However, the operation for setting up the coordinate system using this method entails the problem that the operator's skill makes a difference in setting precision and the risk that the hand and the table may be damaged by erroneous operations since the hand of the robot is brought into physical contact with the table.
The present invention is to provide a system for correcting the operation of a robot using visual feedback, which enables operations on an object involving complicated paths and enables the camera to move parallel to the table surface and rotate about an axis normal to the table surface without an operator's cognition of the robot coordinate system, to thereby efficiently move the camera to a predetermined position.